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^ \ * ^ Office Action Summary 
it ^ 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- tf NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

I) ^ Responsive to communication(s) filed on 06 December 2004 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) IS Claim(s) f-25 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) 13 Claimfe) 1-25 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)S The drawing(s) filed on 06 October 2000 is/are: a)M accepted or b)D objected to by the Examiner. 
Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 185(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

II) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)Q None of: 

1 .□ Certified copies of the priority documents have been received. 
2.D Certified copies of the priority documents have been received in Application No 



3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1 . Applicant's election with traverse of the amplifying circuit in the reply filed on 
12/6/2004 is acknowledged. The traversal is on the ground(s) that all the species 
illustrate an amplifying circuit for a pixel array in an imaging device and that the same 
reference numbers are used through the different embodiments. This is not found 
persuasive because each embodiment has different characteristics that are unique to 
that particular embodiment, which create a burdensome search on the examiner. 
Furthermore, the Applicant has failed to prove that these characteristics would not 
create be a burdensome search. Therefore, the requirement is still deemed proper and 
is therefore made FINAL. 

Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

2. Claims 1-12 are rejected under 35 U.S.C. 102(b) as being anticipated by Dierickx 
etal. (WO 99/16238). 

Regarding claim 1, Dierickx et al. discloses in Fig. 2 an amplifying circuit, 
comprising: an amplifying element (A1) with at least an input terminal and an 
output terminal, a signal input node (11), the signal levels of which at least two 
moments in time are to be amplified by the amplifying element (page 1 1 , lines 6- 
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9), at least two connecting lines between the signal input node and the amplifying 
element (these lines can be seen in Fig. 2), for transferring a signal from the 
signal input node to the input terminal of the amplifying element, a memory 
element (MR1 or MS1) on at least one of the connecting lines, for storing a signal 
level of the signal input node at a moment in time (page 1 1 , lines 1 5-27), a 
switching element disposed on each connecting line (S41 and S51), between the 
memory element and the input terminal of the amplifying element if a memory 
element is provided on the connecting line, for consecutively connecting signal 
levels of the signal input node at different moments in time to the same 
amplifying element, at least one output node (can be seen in Fig. 2), each output 
node being connected to the output terminal of the same amplifying element. 

Regarding claim 2, Dierickx et al. discloses all the limitations as previously 
discussed with respect to claim 1 as well as disclosing that the amplifying circuit 
comprises a memory element on each of the connecting lines (MR1 and MS1). 

Regarding claim 3, Dierickx et al. discloses all the limitations as previously 
discussed with respect to claim 1 including that there are the same number of 
output nodes as there are connecting lines (as can be seen in Fig. 2, there is one 
output line and one connecting line), output nodes and connecting lines being 
associated with each other according to a 1 to 1 relationship, each output node 
being consecutively connected over the same amplifying element to the 
connecting line with which it is associated (this can be seen in Fig 2). 
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Regarding claims 4 and 5, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 1 including that the amplifying element 
is a transistor or a transistor of the type of metal oxide semiconductor transistors. 
(It is inherent that the amplifying element is a transistor or a transistor of the type 
of metal oxide semiconductor transistors). 

Regarding claim 6, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 1 including that the amplifying element 
is an operational transconductance amplifier (It is inherent that the amplifying 
element is an operational transconductance amplifier). 

Regarding claim 7, Dierickx et al. discloses all the limitations as previously 
discussed with respect to claim 1 including that the memory element is a 
capacitor (page 1 1 , lines 20-21 and 27). 

Regarding claim 8, Dierickx et al. discloses in Fig. 2 an array of amplifying 
circuits, each amplifying circuit, comprising: an amplifying element (A1) with at 
least an input terminal and an output terminal, a signal input node (11 ), the signal 
levels of which at least two moments in time are to be amplified by the amplifying 
element (page 1 1 , lines 6-9), at least two connecting lines between the signal 
input node and the amplifying element (these lines can be seen from Fig. 2), for 
transferring a signal from the signal input node to the input terminal of the 
amplifying element, a memory element (MR1 or MS1) on at least one of the 
connecting lines, for storing a signal level of the signal input node at a moment in 
time (page 1 1 , lines 15-27), a switching element disposed on each connecting 



Application/Control Number: 09/684,61 1 Page 5 

Art Unit: 2615 

line (S41 and S51), between the memory element and the input terminal of the 
amplifying element if a memory element is provided on the connecting line, for 
consecutively connecting signal levels of the signal input node at different 
moments in time to the same amplifying element, at least one output node (can 
be seen in Fig. 2), each output node being connected to the output terminal of 
the same amplifying element, the array further comprising: at least one output 
line (Y) in common to all amplifying circuits of the array, the output nodes of the 
amplifying circuits being connected to the output lines. 

Regarding claim 9, Dierickx discloses in Fig. 2 a device for imaging 
applications, comprising: a matrix of active pixels arranged in a geometric 
configuration, each pixel producing an electrical signal indicative of the light 
intensity of a portion of a scene being imaged by that pixel (page 1 , lines 25-32), 
at least one amplifying circuit common to a group of pixels out of the matrix, at 
least one output line wherein each amplifying circuit comprises an amplifying 
element (A1 ) with at least an input terminal and an output terminal, a signal input 
node (11) being intended to obtain electrical signals from pixels out of the group 
of pixels to which the amplifying circuit is common, the signal levels of are to be 
amplified by the amplifying element (page 1 1 , lines 6-9), at least two connecting 
lines between the signal input node and the amplifying element (these lines can 
be seen in Fig. 2), for transferring an electrical signal from the signal input node 
to the input terminal of the amplifying element, a memory element (MR1 or MS1 ) 
on at least one of the connecting lines, for storing a signal level of the electrical 
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signals at the signal input node at a moment in time (Page 1 1 , lines 15-27), a 
switching element disposed on each connecting line, between the memory 
element and the input terminal of the amplifying element if a memory element is 
provided on the connecting line (S41 and S51), for consecutively connecting 
signal levels of the signal input node at different moments in time to the same 
amplifying element, at least one output node (can be seen in Fig. 2), each output 
node being connected to the output terminal of the same amplifying element. 

Regarding claim 10, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 9 including that the matrix is arranged 
in columns and rows and wherein the group of pixels is a row of pixels (Fig. 2). 

Regarding claim 1 1 , Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 9 including that the matrix is arranged 
in columns and rows and wherein the group of pixels is a column of pixels (Fig. 
2). 

Regarding claim 12, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 9 including that the output lines (11) 
are common to the matrix of active pixels, the output node of each amplifying 
circuit being connected to the output lines (Y). 

Regarding claim 13, Dierickx et al. discloses all the limitations as 
previously discussed with respect to any of the claims 9-12 as well as disclosing 
that the device may be used in camera systems or imaging applications requiring 
a high image quality (page 1 , lines 15-19). 
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Regarding claim 14, Dierickx et al. discloses a pixel adapted for 
integration in an imaging device, comprising: a radiation sensitive element able to 
produce an electrical signal indicative of the amount of radiation picked up by 
that pixel (page 1, lines 25-32), an amplifying circuit wherein the amplifying circuit 
comprises an amplifying element (A1 ) with at least an input terminal and an 
output terminal, a signal input node (11), the signal levels of which at least two 
moments in time are to be amplified by the amplifying element the signal levels 
being obtained from the radiation sensitive element (page 1 1 , lines 6-9), at least 
two connecting lines between the signal input node and the amplifying element 
(these lines can be seen in Fig. 2), for transferring a signal from the signal input 
node to the input terminal of the amplifying element, a memory element (MR1 or 
MS2) on at least one of the connecting lines, for storing a signal level of the 
signal input node at a moment in time (page 11, lines 15-27), a switching element 
disposed on each connecting line (S41 and S51), between the memory element 
and the input terminal of the amplifying element if a memory element is provided 
on the connecting line, for consecutively connecting signal levels of the signal 
input node at different moments in time to the same amplifying element, at least 
one output node (can be seen in Fig. 2), each output node being connected to 
the output terminal of the same amplifying element. 

Regarding claim 15, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 14 including that the radiation 
sensitive element is a photodiode (page 1 , lines 25-32). 
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Regarding claim 16, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 14 including that the radiation 
sensitive element is an infrared photodetector (page 1, lines 25-32). 

Regarding claim 17, Dierickx et al. discloses a method for reducing fixed 
pattern noise of solid state imaging device having a group of active pixels (page 
4, lines 30-35), each pixel comprising a radiation sensitive element (page 1, lines 
25-32) and an amplifying circuit, the method comprising the following steps: 
reading out the signal of a pixel brought in a first state and storing the 
corresponding voltage level in a first memory element, reading out the signal of 
the pixel brought in a second state (which is different from the first state) and 
storing the corresponding voltage level in a second memory element (page 1 1 , 
lines 15-28), transferring the signal of the first memory element to an amplifying 
element, amplifying it and transferring it to an output line, transferring the signal 
of the second memory element to the same amplifying element, amplifying it and 
transferring it to an output line, repeating these steps for at least part of the pixels 
of the imaging device (page 1 1 , line 29 - page 1 2, line 8). 

Regarding claim 18, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 1 7 including that the memory element 
uses one output line (can be seen in Fig. 2). 

Regarding claim 19, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 17 as well as disclosing that it further 
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comprises the step of calculating a differential output signal by taking the 
difference between potential values on the output lines (page 3, lines 10-23). 

Regarding claim 20, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 17 including that the first state and the 
second state correspond to different amounts of radiation collected on the 
radiation sensitive element in the pixel (page 11, lines 15-28). 

Regarding claim 21, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 20 including that the first state or 
second state corresponds to an amount of radiation or light collected on the 
radiation sensitive element in the pixel (page 11, lines 15-28). 

Regarding claim 22, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 20 including that the second state or 
the first state corresponds to a non-irradiated or non-illuminated or dark or reset 
state of the pixel (page 1 1 , lines 15-28). 

Regarding claim 23, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 17 including that the pixel is read out 
in additional states and its corresponding voltage level is being stored on 
additional memory elements (page 1 , line 25 - page 4 lines 7; page 1 1 , lines 6 - 
page 12, line 20). 

Regarding claim 24, Dierickx et al. discloses all the limitations as 
previously discussed with respect to claim 17 including that the signal of the first 
memory element is transferred to the first output line common for the group, and 
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concurrently, the signal of the second memory element of another amplifier is 
transferred to the second output line common for the group (Fig. 3). 

Regarding claim 25, Dierickx discloses a method for reducing fixed pattern 
noise and kTC noise in a solid state imaging device having a group of active 
pixels (page 10, lines 29-35), each pixel comprising a radiation sensitive element 
(page 1, lines 25-32) and an amplifying circuit, the method comprising the 
following steps: reading out the signal of a pixel brought in a first state, 
corresponding to the non-illuminated or dark condition of the pixel or to the reset 
state of the pixel, and storing the corresponding voltage level alternatingly on a 
first or a third memory element, reading out the signal of the pixel in a second 
state, at a later moment in time, corresponding to an amount of radiation or light 
collected on the radiation sensitive element on the pixel, and storing the 
corresponding voltage level on a second memory element alternatingly (page 1 1 , 
lines 15-27), transferring the signal of the first or third memory element to an 
amplifying element, amplifying it and transferring it to an output line that is 
common to the group of pixels, transferring the signal of the second memory 
element to the same amplifying element, amplifying it and transferring it to an 
output line that is common to the group of pixels, repeating this operation for 
essentially all or part of the pixels of the imaging device (page 1 1 , line 29 - page 
12, line 8). 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Heather R. Long whose telephone number is 571-272- 
7368. The examiner can normally be reached on Mon. - Thurs.: 7:00 am - 4:30 pm, and 
every other Fri.: 7:00 am - 3:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, James Groody can be reached on 571-272-7950. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 
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pgyiCBS AflD methods fqr iwpRQVffTG th e jmaqe; qtjai^ity AW 

Field of the invention 

The present invention relates to solid state 
imaging devices being manufactured in a CMOS- or MOS- 
technology. 

15 More particularly, the principal object of 

the present invention is related to methods and devices 
which are able to improve the image quality in an image 
sensor . 

Another object of the present invention is 
20 related to the improvement of the image quality by a method 
of correcting isolated pixel values present in an image 
taken by imaging devices. 

BftckgyQifflfl of the invention 

25 Solid state image sensors are well known. 

Virtually all solid-state imaging sensors have as key 

element a photosensitive element being a photoreceptor, a 

photo diode, a photo transistor, a CCD gate, or alike. 

Typically, the signal of such a photosensitive element is a 
30 current which is proportional to the amount of 

electromagnetic radiation (light) falling onto the 

photosensitive element. 

A structure with a photosensitive element 

included in a circuit having accompanying electronics is 
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called a pixel . Such pixel can be arranged in an array of 
pixels so as to build focal plane arrays of rows and 
columns . 

Commonly such solid state image sensors are 
5 implemented in a CCD- technology or in a CMOS- or MOS- 
technology. Solid state image sensors find a widespread use 
in devices such as camera systems. In this application a 
matrix of pixels comprising light sensitive elements 
constitutes an image sensor, which is mounted in the camera 

10 system. The signal of said matrix is measured and 
multiplexed to a so-called video- signal . 

Of the image sensors implemented in a CMOS- 
or MOS -technology, CMOS or MOS image sensors with passive 
pixels and CMOS or MOS image sensors with active pixels are 

15 distinguished. An active pixel is configured with means 
integrated in the pixel to amplify the charge that is 
collected on the light sensitive element. Passive pixels do 
not have said means and require a charge sensitive 
amplifier that is not integrated in the pixel. For this 

20 reason, active pixel image sensors are potentially less 
sensitive to noise fluctuations than passive pixels. Due to 
the additional electronics in the active pixel, an active 
pixel image sensor may be equipped to execute more 
sophisticated functions, which can be advantageous for the 

25 performance of the camera system. Said functions can 
include filtering, operation at higher speed or operation 
in more extreme illuminations conditions. 

Examples of such imaging sensors are 
disclosed in EP-A-0739039, in EP-A-0632930 and in 

30 US-A-5608204 . The imaging devices based on the pixel 
structures as disclosed in these patent applications 
however are still subject to deficiencies in the image 
quality of the devices. 
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A problem in these CMOS based imaging devices 
appears because material imperfections and technology 
variations have as effect that there is a non- uniformity in 
the response of the pixels in the array. This effect is 
5 caused by a non- uniformity or fixed pattern noise (FPN) or 
by a photoresponse non-uniformity (PRNU) . Correction of the 
non-uniformity needs some type of calibration, e.g. by 
multiplying or adding /subtracting the pixel's signals with 
a correction amount that is pixel -dependent . 

10 Several methods to cancel FPN are based on 

techniques that are often called correlated double sampling 
or offset compensation. The methods in general are based on 
the following: the signal of the pixel is subtracted from 
the signal of the same pixels in a reference state (this 

15 reference state is typically the reset or dark state) . The 
difference of both signal is free of pixel -related non- 
uniformity. However, if the differencing circuit is common 
for a part of the imager (typically, common for one 
column) , a new non-uniformity will originate due to the 

20 non-uniformity of the differencing circuits. In a typical 
APS imager with common column buffers or column amplifiers, 
the new fixed pattern noise is column dependent, and is 
visible in the image as a shade of vertical stripes. 

A stripe -shaped FPN is much more annoying. 

25 than a pure statistical FPN. It is seen in experiments that 
a true random FPN of 5% RMS is barely visible to the human 
eye, whereas a stripe-shaped FPN remains visible even when 
the amplitude is below 1% RMS. The reason is that the human 
eye has a kind of built-in spatial filter that recognises 

30 larger structures even when they have low contrast. 

Even if in the case that we have no fixed 
pattern noise, the photoresponse non-uniformity can be 
different from 0. 
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Another problem arises due to processing 
imperfections, statistics, etc. This means that typically, 
a finite number of pixels in a pixel array will be 
defective (hard faults) or yield a signal that deviates 
5 visibly from the "exact" pixel value. Such faults appear as 
white or black (or grey) points in the image. This class of 
faults in the sequel is called an isolated pixel value. 

A known way to cancel these spots is to store 
a list of them and of their positions in the image in a 

10 memory unit in the device. In an image processing step, the 
isolated pixel value is then replaced by e.g. the average 
of the surrounding pixels. This method is viable, but has 
the disadvantage that it requires a memory. Moreover, it 
cannot handle isolated pixel values that appear 

15 intermittently or only in certain cases. A good example, is 
a so-called dark current pixel. Such pixels will appear 
white in a dark environment, but will behave normal in a 
bright environment. 

Other ways to cancel isolated pixels faults 

20 have been proposed, e.g. the spatial median filter or other 
types of Kalman filters can be used to remove such isolated 
faults. Unfortunately, such filters do also remove useful 
detail from the image. Consider the image of a star covered 
sky with an image sensor that has some faulty pixels that 

25 appear white. The quoted filters are not able to remove the 
white point due to faults, and leave the white points that 
are stars untouched. 
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30 The present invention aims to suggest a pixel 

structure and methods to improve the image quality, more in 
particular the image non-uniformity of in array of pixels 
by cancellation of the appearance of column- shaped fixed 
pattern noise (FPN) . 
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Main cha racteristics of the present invention 

As a first object, the present invention is 
related to an image sensor comprising an array of rows and 
columns of pixels, all the pixels of one column of the 
array being connected to at least one common pixel output 
line having at least one memory element and at least a 
first amplifying element, all these amplifying elements 
being connected to a common output amplifier. 

According to one preferred embodiment, the 
image sensor further comprises : 

— a second amplifying element on the output of the memory 
element , 

— said common output amplifier having at least a first and 
a second input terminals, 

— means for switching the pixel 1 s signal on the common 
output line and the memory element's signal to 
respectively first and second amplifying elements of one 
column, and 

— means for switching the two output signals of the 
amplifying elements of one column to respectively first 
and second input terminals of said common output 
amplifier. 

Preferably, the switching means comprise at 
least one cross-bar switch. 

According to another preferred embodiment, 
the image sensor further comprises before the amplifying 
element two parallel circuits being connected through 
switches to the common pixel output line, at least one 
circuit having said memory element. Preferably, both 
circuit have a memory element. 

According to another preferred embodiment, 
said common pixel output line is being connected through 
switches to said memory element and said amplifying 
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element, where the offset of the amplifying element is 
stored on the memory element during a first phase of the 
read-out, and this offset is subtracted from the signal of 
the amplifying element during the second phase of the read- 
5 out. 

The present invention is also related to a 
method of reading out an array of rows and columns of 
pixels in an image sensor as described hereabove according 
to the first embodiment, comprising the steps of: 
10 - amplifying the output signals of essentially each pixel 
of one column in the first amplifying element thereby 
obtaining a set of amplified pixel output signals, 

- amplifying the reference signals of essentially each 
pixel of one column in the second amplifying element, 

15 thereby obtaining a set of amplified pixel reference 
signals, 

- consecutively, for essentially each pixel of said column 
imposing the amplified pixel output signal to a first or 
a second terminal of said common output amplifier and 

20 imposing the amplified pixel reference signal to a 
second or a first terminal of said common output 
amplifier, while switching the amplified pixel output 
signal to respectively said first and said second 
terminals for essentially each consecutive pixel of said 
25 column, said reference signal being imposed to the other 
terminal of said common output amplifier. 

A voltage can be imposed to a node. In this 
case, it means e.g. that the node is connected to a voltage 
source. The voltage source should be higher than the node, 
30 i.e. it should have a lower impedance. 

According to another preferred embodiment, 
the present invention is related to a method of reading out 
an array of rows and columns of pixels in an image sensor 
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as described hereabove in the second embodiment of the 
present invention, comprising the steps of : 

- sampling the signal in a first phase and storing it in a 
memory element, 

5 - sampling the signal in a second phase and possibly 
storing it in another memory element, 

- subtracting said first signal from said second signal in 
a unique output circuit. 

According to another preferred embodiment, 
10 the present invention is related to a method of reading out 
an array of rows and columns of pixels in an image sensor 
as described in the third embodiment of the present 
invention, comprising the steps of : 

- during a first phase, calibrating the output of the 
15 amplifying element to a predetermined value, 

- storing said value in a memory element during the 
application of a first signal of said pixel on the line, 

- during a second phase, applying a second signal of said 
pixel on the line in order to have on the output a signal 

20 proportional to the difference between first and second 
signals. 

Another aspect of the present invention is 
related to a method of replacing an isolated pixel value in 
the image of an image sensor, being an array of pixels, and 
25 wherein at least one current source is connected to the 
pixels, the method comprising the step of : 

- limiting said isolated pixel value between or to an 
upper and/ or a lower bound that is derived from the 
values of pixels in the immediate neighbourhood of the 

30 said isolated pixel value. 

Preferably, said upper and/or lower bounds 
are found by extrapolating the immediate neighbourhood 
pixel values towards a value that corresponds to the 



WO 99/16238 PCT/BE98/00139 

8 

position of said individual pixel in relation to the 
immediate neighbourhood pixels. 

Said upper and/or lower bounds are found by 
extrapolating the values of a neighbour (V^N) of the pixel 
5 having said isolated pixel value and of the neighbour 
thereafter {V2N) , the replacing pixel value being 
calculated as V^N+n* (V1N-V2N) , n being a real number. 

The neighbourhood and the neighbourhood 
thereafter are on the same row of said array.. Preferably, 
10 the upper bound is the maximum of a set of values, said set 
being determined as the pixel values (a,b,c,d,e) of pixels 
on the same row of said array as said isolated pixel, said 
upper bound being calculated as 

cmax = F(a,b,c,d), or cmax G(E(a,b), E(e,d), E(b), E(d)) 
15 where F is a non-linear or linear function, G is a non- 
linear GE or linear function, E is an extrapolating 
function, wherein cmax= MAX(2b-a, 2d-e,b, d) 
together with 

cmin= MIN(2b-a,2d-e,b,d) 
20 with MAX () yielding the maximum, respectively the minimum 
of the arguments, the corrected c-value being obtained as 
c=MIN (MAX (c , cmin) , cmax) . 

25 prjgf jtegcript&Qii og tto 4yawinqp 

Figure la represents a particular implementation of a 
column FPN cancellation method and the 
corresponding image sensor structure 
therefor . 

30 Figure lb represents an embodiment of a cross-bar 

switch used in the structure represented in 
Fig. la. 
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Figures 5 



Figure 2 describes another particular implementation 

of a column FPN cancellation method and the 
corresponding image sensor structure 
therefor. 

describes a particular embodiment of an 
output block being used in the structure as 
represented in Fig. 2. 

represents the switching diagram for the 
read-out signal applied to the several 
switches in the structure as represented in 
Fig . 2 . 

represent another particular implementation 
of a column FPN cancellation method and the 
corresponding image sensor structure 
therefor, wherein Fig. 5a represents more 
particularly one column in an array of pixels 
being connected to two different structures 
represented in details in Fig. 5b and 5c 
during a first phase and a second phase, 
represents the specific topology used for the 
structure represented in Fig. 5b and 5c. 
represents the switching diagram for the 
read-out sequence to the several switches 
used in the structure represented in Fig. 6. 
Figures 8a and 8b represent a method of correcting 
isolated white pixel values being present in 
an image composed by an array of pixels. 



20 Figure 6 
Figure 7 

25 



Brief description of preferrftrf ftmHr>dimenta of the present 
30 invention 

As a first object of the present invention, a 
first structure of an APS image sensor is represented in 
Fig. 1. References (X 1; j) , (X 2 j) and (X 3 j) are three pixels 



\ 
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of a column of an image sensor. The pixel's signal on a 
common output line (lj) is represented in particular by the 

column bus 11 K" in the present case and is fed to the 
optional buffer amplifier Aj , and/or stored on a memory 
5 element (capacitor C + switch S3) , and fed to amplifier B j . 

By the relative timing of the addressed pixel 1 s reset pulse 
and the control of the switch S3, one can make that the 
pixel's signal and its reference level are available on 
amplifiers Aj, reps. B j . The fact that the signal passes 
10 through the column amplifiers Aj and B j , is a source of 

offset non- uniformity, which is column related and causes a 
vertical stripe-* shaped FPN. More specifically, each column 
will feature an average "OV" offset voltage referred to the 
average of the other columns. 

15 Switches (SI) and (S2) are crossbar switches. 

Suppose that they are in the forward direction either in 
crossed directions. Both switches SI and S2 operate 
synchronously. In both cases, the signal on the capacitor C 
goes to the input of the output amplifier, and the signal 

20 on K goes to the + input of the output amplifier. Yet, the 
"OV" of the column will be positive in the one case and 
negative in the other case. Since the switches (S1)/(S2) 
are modulated, e.g. essentially turned direction at each 
new row (i = 1, 2, 3) of the image, the average offset of a 

25 column will be zero. For each individual pixel of a column, 
there will be indeed remain an offset which is + or - OV 
but this is a very small feature, and is not recognised by 
the eye as a stripe. 

Another embodiment of the present invention 

30 is to suggest a read-out scheme for image sensors that 
suppresses the effects of non- uniformities caused by 
variations in pixels as well as variations in the output 
amplifiers. This read-out scheme can be used in sensors 
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that provide the output signal in terms of a difference. 
For example, in a sensor with integrating pixels, this 
difference is the voltage between the output when the pixel 
is on reset state and the output voltage after integration 
5 time. 

Namely the method suggests to subtract the 
signal when a pixel is reset from the signal after the 
integration time, in order to have a signal which is free 
from pixel variations. In order to avoid the introduction 

10 of the non-uniform column amplifiers effects, the signal of 
the reset state as well as the signal after integration are 
sampled and held by the column read-out circuit. Finally, 
the subtraction is being carried out by a unique subcircuit 
at the sensor 1 s output (D) . This is detailed in Fig . 2 . 

15 For every row of pixels the read-out process 

is, of course, identical. Let us assume that the i tl1 row is 
selected. When pixels are reset, the switches controlled by 
the signal Ol, are closed, thus the reset -level output of 

every pixel on the it* 1 row, namely % , is stored on the 
20 corresponding memory element M r j (which is in the present 
case a capacitor) . 

Then, the switches S4j controlled by Ol are 

opened and pixels start integrating the charge carriers 
produced by the impinging light. After the lapse of the 
25 integration time, the switches S5j controlled by <D2 are 

closed, thus storing the values of the pixel output to the 
memory element M s j (also a capacitor) . This value, for the 

pixel with coordinates i j is denoted as jp* . 

After the sample and hold phase for the two 
30 voltages, % and % , for the first column and by 
proceeding from the first column to the last , the 
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appropriate column read-out circuit is connected with the 
output . 

When the i th column has been selected, the 
signal 03 drives the switch S6j to led the signal % to 

5 the output modified according to the action of the column 
amplifier, so as a signal 

y 9 =AjX 9 +Bj <n 

is led to block D (common output amplifier) . 

The block D is an easily realisable circuit 
10 with an output 

z(n) = [y(n) - y(n-l)] (2) 
where y(n) denotes the input as instant n. 

A preferred embodiment of such output block 
(D) is described in details in Fig. 3. Therefore, the 
15 output signal will be free from variations in the 
characteristics of pixels and the column amplifiers, 

j4 - 4. *;-*;) o) 

where Aj is easily reproducible, by example by using source 
followers as the local final phase of the column circuits 
20 (when Aj = 2 , B j = ~ v thj ) • 

Fig. 4 shows a switching diagram for the 
above-mentioned read-out circuit wherein 03 is controlling 
S4 j , 04 is controlling the switch S5j and <E>5 is 
controlling the switch S6j . 

25 According to another embodiment of the 

present invention, an attempt to overcome the problem of 
offset introduced by the column in a image sensor 
consisting of pixels is described in Figs. 5 & 6, which can 
be used with pixels that are read-out twice in every 

30 access. For example, in integrating pixels, one read 
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operation is being performed when the pixel is set on the 
reset phase (first phase) and the second read-out moment is 
after a certain integration time (second phase), the first 
phase is defined by a period wherein the read-out signal of 
5 the pixel is according a first state while the second phase 
is defined by a period such as an integration period where 
the read-out signal of the pixel is in another state. 

In the first phase, when pixel output is V^, 

the capacitor C stores a charge : 
10 Q = C(V ac -V out ) = C(V ac -V 1+ V t h) (4) 
where V^h is the threshold voltage of p. 

During second phase, when pixel output is V 2 , 

the capacitor stores again charge Q which now can be 
expressed as : 

15 Q = C(V g/i -V 2 ) (5) 
where V g ^ is the gate voltage of p. 

From (4) and (5) , we obtain : 
Vg M - V 2 -V 1+ V ac+ V th (6) 
Therefore : 

20 V out « V 2 -V 1+ V ac (7) 

i.e. the output voltage does not depend on the V^h (where 
variations in the V t h cause offsets in the signal V out ) . 

The column amplifier can be implemented by 
using the topology described in details in Fig. 6. 
25 Switches S7, S8, S9, S10 and Sll are being 

controlled by signals C>5 and OS (when C>i is high, the 
corresponding switches are closed) as represented in 
Fig . 7 . 

According to a preferred embodiment 
30 represented in Fig. 7, S7, S8 and S9 are controlled by OS 
while S10 and Sll are controlled by <t>6 (when <M is high, 
the corresponding switches are closed) . This means that <D5 
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is describing the situation represented in Fig. 5b while 
06 is representing the situation represented in Fig, 5c. 

According to another aspect, the present 
invention is able to discriminate between isolated pixel 
5 faults and features in the real image. In the case of an 
image of a star covered sky, it should be noted that the 
fact that the image projected through a lens is never 
perfectly sharp. Even with good lenses, a star image is not 
projected on a single pixel. Always the point like source 

10 of the start will be smeared out over a central pixel and a 
few neighbours. In a 1-dimensional cross section, a star 
image will look like the image in Fig. 8a, while an 
isolated pixel fault will look like in Fig. 8b. 

In the above simple example, the peak in 

15 Fig. 8b should be removed, while the peak in 8a should 
remain untouched. 

The advantage is clear, only device faults 
are corrected while normal images are left untouched. The 
operation causes no visible image degradation in faultless 

20 parts of the image. 

According to this second aspect of the 
present invention, a method to remove an isolated whiter or 
darker pixel from the image is suggested. This method 
consists in limiting the value of every individual pixel 

25 between an upper and/or a lower bound that is/are derived 
from the values of pixels in the intermediate neighbourhood 
of the said pixel. 

Preferably, the upper and/or lower bounds are 
found by extrapolation of the neighbourhood pixel values 

30 towards the position of the said individual pixel. The 
upper and/or lower bound are/is a combination of one or 
several such 1-D or 2-D extrapolations done with different 
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methods, and/or from different sides of the said individual 
pixel . 

Preferably, extrapolation is the linear 
extrapolation of a neighbour (Nl) of the said individual 
5 pixel (IP) and the neighbour thereafter (N2) . The 
extrapolated value is calculated as 2*N1-N2 or more 
general: Nl + n* (N1-N2) where the parameter n is a real 
number, typically between 0 and 3. 

According to another preferred embodiment, 
10 the calculation of the upper bounds is performed by 
extrapolating values from the two sides of said individual 
pixels. The advantage is that only the pixels data in 1 
line of an image are required, which saves memory and 
operations and allows straightforward implementation as a 
15 pipelined filter. Also such a filter is able to correct a 
vertical line defect. 

In the example of Figs. 8a and 8b, five 
pixels in a neighbourhood (a 5-pixel "kernel") are taken. 
The experience is that smaller kernels do not yield good 
20 results. Larger kernels may give some improvements compared 
to the 5 -pixel kernel. 
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CLAIMS 

1. An image sensor comprising an array of 
rows (i) and columns (j) of pixels (X^j), all the pixels of 
one column of the array being connected to at least one 

5 common pixel output line (lj) having at least one memory 

element (Mj) and at least a first amplifying element (Aj), 
all these amplifying elements (Aj) being connected to a 

common output amplifier (D) , characterised in that the 
sensor further comprises: 
10 - a second amplifying element (Bj) on the output of the 
memory element (M j ) , 

- said common output amplifier (D) having at least two 
input terminals, 

- means (SI) for switching the pixel's signal on the common 
15 output line (lj) and the memory element's signal (Mj) to 

respectively third and second amplifying elements (Aj and 

Bj) of one column, and 

- means (S2) for switching the two output signals of the 
amplifying elements (A j , Bj) of one column to 

20 respectively first and second input terminals of said 
common output amplifier (D) . 

2 . An image sensor as recited in claim 1 , 
wherein said switching means comprise at least one cross- 
bar switch. 

25 3 . An image sensor comprising an array of 

columns and rows of pixels (Xi j ) , all the pixels of one 

column of the array being connected to at least one common 
pixel output line (lj) having at least one memory element 
(Mj) and at least one amplifying element (Aj) r all these 
30 amplifying elements (Aj) being connected to a common output 
amplifier (D) , characterised in that before the amplifying 
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element (Aj) # the common pixel output line (lj) is divided 
through switches (S4j and S5j) in two parallel circuits, at 
least one circuit having said memory element (Mj ) . 

4. An image sensor as recited in claim 3, 
5 wherein both circuits have a memory element (Ms j and Mr j ) . 

5* An image sensor comprising an array of 
columns and rows of pixels (X-jj), all the pixels of one 

column of the array being connected to at least one common 
pixel output line (lj) having at least one memory element 

10 (Mj) and at least one amplifying element (Aj), all these 
amplifying elements (Aj) being connected to a common output 
amplifier (D) , characterised in that said common pixel 
output line (lj) is being connected through switches (S7j, 
S8j, S9j and S10 j , Sllj) to a memory element (Cj) and an 

15 amplifying element (/ij)r where the offset of the amplifying 
element is stored on the memory element during a first 
phase of the read-out, and this offset is subtracted from 
the signal of the amplifying element during the second 
phase of the read- out . 

20 6. A method of reading out an array of rows 

and columns of pixels (X^j) of an image sensor according to 
claim 1, comprising the steps of : 

- amplifying the output signals of essentially each pixel 
of one column in the first amplifying element (Aj) 

25 thereby obtaining a set of amplified pixel output 
signals, 

- amplifying the reference signals of essentially each 
pixel of one column in the second amplifying element 
(Bj), thereby obtaining a set of amplified pixel 

30 reference signals, 

- consecutively, for essentially each pixel (i = 1, 2, 3) 
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of said column imposing the amplified pixel output 
signal to a first or a second terminal of said common 
output amplifier (D) and imposing the amplified pixel 
reference signal to a second or a first terminal of said 
5 common output amplifier (D) , while switching the 
amplified pixel output signal to respectively said first 
and said second terminals for essentially each 
consecutive pixel of said column, said reference signal 
being imposed to the other terminal of said common 
10 output amplifier. 

7. A method of reading out an array of rows 
and columns of pixels (Xij) of an image sensor as recited 
in claim 3, comprising the steps of : 

- sampling the signal in a first phase and storing it in a 
15 memory element (Mi j ) , 

- sampling the signal in a second phase and possibly 
storing it in another memory element, 

- subtracting said first signal from said second signal in 
a unique output circuit (D) . 

20 8. Method as recited in claim 7, wherein said 

first phase is the reset phase and said second phase is 
after the integration period. 

9. A method of reading out an array of rows 
and columns of pixels (X^j) of an image sensor as recited 

25 in claim 5, comprising the steps of : 

- during a first phase, calibrating the output of the 
amplifying element to a predetermined value, 

- storing said value in a memory element during the 
application of a first signal of said pixel on the line, 

30 - during a second phase, applying a second signal of said 
pixel on the line in order to have on the output a signal 
proportional to the difference between first and second 
signals. 
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